Skip to content

Add a sample buffer renderer output for AirPlay 2 multi-room - #158

Open
chrizzchrizz wants to merge 1 commit into
dimitris-c:mainfrom
chrizzchrizz:feat/sample-buffer-output
Open

chrizzchrizz wants to merge 1 commit into
dimitris-c:mainfrom
chrizzchrizz:feat/sample-buffer-output

Conversation

@chrizzchrizz

Copy link
Copy Markdown
Contributor

Add a sample buffer renderer output for AirPlay 2 multi-room

Problem

On Mac Catalyst the system route picker (AVRoutePickerView / MPVolumeView) lets the user pick one or more AirPlay 2 devices, for example two rooms at once. When the player uses an output audio unit (AVAudioEngine), picking AirPlay devices does not change the audio session route or the engine configuration, and the output just goes silent. Restarting the engine does not help. Only an AVSampleBufferAudioRenderer follows the devices picked there. Combined with the .longFormAudio route sharing policy, it also plays to several rooms at once.

Changes

A new, opt-in output mode in AudioPlayerConfiguration:

public enum AudioPlayerOutputMode: Equatable, Sendable {
    case audioUnit                                       // default, unchanged
    case sampleBufferRenderer(maxBufferedSeconds: Double)
}

With .sampleBufferRenderer, no output audio unit is created. Instead, the new SampleBufferRendererOutput uses a timer to pull the engine's rendered audio (AudioPlayerRenderProcessor.renderOutput). It enqueues the audio into an AVSampleBufferAudioRenderer, which an AVSampleBufferRenderSynchronizer drives.

The renderer plays audio enqueued up to maxBufferedSeconds ahead of what is heard, so the player compensates for that latency:

  • progress, duration and framesPlayed report the position that is heard. The previous item is reported until its end has been heard.
  • The item change callbacks (didStartPlaying, didFinishPlaying) are delayed until the change can be heard, and they keep their order (OrderedDelayedNotifier).
  • Seek and a new item flush the renderer, and pending callbacks are delivered at once. At the end of the queue the renderer drains instead.
  • When the renderer flushes automatically (AVSampleBufferAudioRendererWasFlushedAutomatically, e.g. on a route change), the audio is enqueued again from a short history, starting at the flush time.
  • volume and muted act on the renderer directly, so they apply at once and no muted audio stays enqueued.

The default output mode stays .audioUnit. Every new code path depends on the renderer output being set, so the existing output is unchanged.

Notes

  • flush(fromSourceTime:) of the renderer always failed in my tests (local output and AirPlay), so a seek uses a full flush. With AirPlay this gives a short rebuffer pause after the seek, which seems to be inherent to AirPlay.
  • This adds no unit tests: the new code needs real audio output and AirPlay devices. Happy to add tests if you have an approach in mind.

Testing

  • swift test: all tests pass. Builds for macOS, iOS and tvOS.
  • Used in a Mac Catalyst music player with .sampleBufferRenderer(maxBufferedSeconds: 4) and the .longFormAudio policy: playback on the Mac speakers, on one AirPlay room and on two rooms at once, with seek, pause/resume, item changes and the end of the queue. On iOS the app keeps the default .audioUnit output.

New output mode `.sampleBufferRenderer(maxBufferedSeconds:)`: instead of an
output audio unit, a timer pulls the engine's manually rendered audio and
enqueues it into an AVSampleBufferAudioRenderer driven by an
AVSampleBufferRenderSynchronizer. On Mac Catalyst only this output follows
the AirPlay devices picked in the system route picker, including several
rooms at once (with the long-form audio route sharing policy).

The audio is enqueued ahead of what is heard, so progress, duration and the
item change callbacks are compensated by the unheard audio content: the
previous item is reported until its end is heard, and the callbacks are
delivered in order once the change is audible. Seek and item changes flush
the renderer, the end of the queue drains, automatic flushes re-enqueue
from a history, volume and mute act on the renderer directly.

The default output mode stays `.audioUnit`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant